Deep Reinforcement Learning for Adaptive Parameter Control in Differential Evolution for Multi-Objective Optimization

Evolutionary algorithms (EA) are efficient population-based stochastic algorithms for solving optimization problems. The performance of EAs largely depends on the configuration of values of parameters that control their search. Previous works studied how to configure EAs, though, there is a lack of a general approach to effectively tune EAs. To fill this gap, this paper presents a consistent, automated approach for tuning and controlling parameterized search of an EA. For this, we propose a deep reinforcement learning (DRL) based approach called ‘DRL-APC-DE’ for online controlling search parameter values for a multi-objective Differential Evolution algorithm. The proposed method is trained and evaluated on widely adopted multi-objective test problems. The experimental results show that the proposed approach performs competitively to a non-adaptive Differential Evolution algorithm, tuned by grid search on the same range of possible parameter values. Subsequently, the trained algorithms have been applied to unseen multi-objective problems for the adaptive control of parameters. Results show the successful ability of DRL-APC-DE to control parameters for solving these problems, which has the potential to significantly reduce the dependency on parameter tuning for the successful application of EAs

Operator Selection in Adaptive Large Neighborhood Search using Deep Reinforcement Learning

Large Neighborhood Search (LNS) is a popular heuristic for solving combinatorial optimization problems. LNS iteratively explores the neighborhoods in solution spaces using destroy and repair operators. Determining the best operators for LNS to solve a problem at hand is a labor-intensive process. Hence, Adaptive Large Neighborhood Search (ALNS) has been proposed to adaptively select operators during the search process based on operator performances of the previous search iterations. Such an operator selection procedure is a heuristic, based on domain knowledge, which is ineffective with complex, large solution spaces. In this paper, we address the problem of selecting operators for each search iteration of ALNS as a sequential decision problem and propose a Deep Reinforcement Learning based method called Deep Reinforced Adaptive Large Neighborhood Search. As such, the proposed method aims to learn based on the state of the search which operation to select to obtain a high long-term reward, i.e., a good solution to the underlying optimization problem. The proposed method is evaluated on a time-dependent orienteering problem with stochastic weights and time windows. Results show that our approach effectively learns a strategy that adaptively selects operators for large neighborhood search, obtaining competitive results compared to a state-of-the-art machine learning approach while trained with much fewer observations on small-sized problem instances

Learning Adaptive Evolutionary Computation for Solving Multi-Objective Optimization Problems

Multi-objective evolutionary algorithms (MOEAs) are widely used to solve multi-objective optimization problems. The algorithms rely on setting appropriate parameters to find good solutions. However, this parameter tuning could be very computationally expensive in solving non-trial (combinatorial) optimization problems. This paper proposes a framework that integrates MOEAs with adaptive parameter control using Deep Reinforcement Learning (DRL). The DRL policy is trained to adaptively set the values that dictate the intensity and probability of mutation for solutions during optimization. We test the proposed approach with a simple benchmark problem and a real-world, complex warehouse design and control problem. The experimental results demonstrate the advantages of our method in terms of solution quality and computation time to reach good solutions. In addition, we show the learned policy is transferable, i.e., the policy trained on a simple benchmark problem can be directly applied to solve the complex warehouse optimization problem, effectively, without the need for retraining

Learning to Adapt Genetic Algorithms for Multi-Objective Flexible Job Shop Scheduling Problems

The configuration of Evolutionary Algorithm (EA) parameters is a significant challenge. While previous studies have examined methods for configuring EA parameters, there remains a lack of a general solution for optimizing these parameters. To overcome this, we propose DEMOCA, an automated Deep Reinforcement Learning (DRL) method for online control of multi-objective EA parameters. When tested on a multi-objective Flexible Job Shop Scheduling Problem (FJSP) using a Genetic Algorithm (GA), DEMOCA was found to be as effective as grid search while requiring significantly less training cost

Combining Deep Reinforcement Learning with Search Heuristics for Solving Multi-Agent Path Finding in Segment-based Layouts

A multi-agent path finding (MAPF) problem is concerned with finding paths for multiple agents such that certain objectives, such as minimizing makespan, are reached in a conflict-free manner. In this paper, we solve a practical MAPF problem with automated guided vehicles (AGVs) for the conveying of luggage in segment-based layouts (MAPF-SL).Most existing algorithms for MAPF are mainly focused on grid environments. However, the conflict prevention problem is more challenging with segment-based layouts in which software is constrained to oversee that vehicles remain on predefined travel paths. Hence, the existing multi-agent path finding algorithms cannot be applied directly to solve MAPF-SL. In this paper, we propose an algorithm, called WHCA*S-RL, that combines Deep Reinforcement Learning (DRL) with a heuristic approach for solving MAPF-SL. DRL is used for determining travel directions while the heuristic approach oversees the planning in a segment-based layout. Our experiment results show that the proposed WHCA*S-RL approach can be successfully used for making path plans in which traffic congestion is both avoided and relieved. In this way, individual vehicles are found to reach goal destinations faster than the approach with search only

Transgressive Behavior in Dutch Youth Sport

The current article reports on the second large-scale prevalence study on transgressive behavior in sport in the Netherlands, and is a follow up of an earlier, comparable prevalence study in 2015. Using a dedicated and customized online questionnaire, approximately 4000 adults who met the inclusion criteria (18 to 50 years old and have played sports in an organized context during childhood in the Netherlands) were surveyed with respect to their experiences of childhood psychological, physical, and sexual transgressive behavior while playing sports. The survey showed that 71.7% experienced some form of transgressive behavior as a child, in which 48.6% of these events also made an impact (in other words, was significant at the time it took place). The degree of impact the event made is also related to the severity of the event. Severe emotional transgression events occurred in 22% of the youth athletes, severe physical assault events in 12.7%, and severe sexual assault events occurred in 6.9% of the youth athletes. Disabled athletes, and those competing at national and international levels, report more experiences of transgressive behavior in sport. The results are consistent with former research and indicate the need for structural attention to create a safe sports climate

Transgressive Behavior in Dutch Youth Sport

The current article reports on the second large-scale prevalence study on transgressive behavior in sport in the Netherlands, and is a follow up of an earlier, comparable prevalence study in 2015. Using a dedicated and customized online questionnaire, approximately 4000 adults who met the inclusion criteria (18 to 50 years old and have played sports in an organized context during childhood in the Netherlands) were surveyed with respect to their experiences of childhood psychological, physical, and sexual transgressive behavior while playing sports. The survey showed that 71.7% experienced some form of transgressive behavior as a child, in which 48.6% of these events also made an impact (in other words, was significant at the time it took place). The degree of impact the event made is also related to the severity of the event. Severe emotional transgression events occurred in 22% of the youth athletes, severe physical assault events in 12.7%, and severe sexual assault events occurred in 6.9% of the youth athletes. Disabled athletes, and those competing at national and international levels, report more experiences of transgressive behavior in sport. The results are consistent with former research and indicate the need for structural attention to create a safe sports climate

The Ongoing Battle Between Infrapopliteal Angioplasty and Bypass Surgery for Critical Limb Ischemia

Background: Critical limb ischemia (CLI) represents the extreme of the peripheral arterial occlusive disease spectrum and is associated with high mortality. Limb salvage often requires infrapopliteal revascularization by either angioplasty or bypass surgery. The past decade has witnessed a paradigm shift in CLI management toward endovascular treatment. This narrative review describes the clinical outcome, treatment strategy, and limitations of both modalities. Method: A literature search was performed of the PubMed and Cochrane databases. All articles, published until September 2011, describing treatment by infrapopliteal arterial revascularization were included. Results: Angioplasty and bypass surgery are both related to a limb salvage rate of approximately 80% at 3-year follow-up. Patency rates appear to be higher after surgery. A reliable comparison of the two modalities, however, is complicated by various confounders, including patient selection, lesion characteristics, and complication rates. Additionally, most studies did not describe the standard use of best medical treatment or outcome for relief of ischemic pain, wound healing, or functional improvement. Conclusion: Infrapopliteal angioplasty and bypass surgery both provide an acceptable limb salvage rate, but patency appears to be better after bypass surgery. Both modalities are likely to be complementary. Additional randomized trials are indicated to provide a treatment algorithm for patients with CLI and infrapopliteal arterial occlusive disease

Endarterectomy or carotid artery stenting:the quest continues

Background: Carotid endarterectomy (CEA) is still considered the "gold-standard" of the treatment of patients with significant carotid stenosis and has proven its value during past decades. However, endovascular techniques have recently been evolving. Carotid artery stenting (CAS) is challenging CEA for the best treatment in patients with carotid stenosis. This review presents the development of CAS according to early reports, results of recent randomized trials, and future perspectives regarding CAS. Methods: A literature search using the PubMed and Cochrane databases identified articles focusing on the key issues of CEA and CAS. Results: Early nonrandomized reports of CAS showed variable results, and the Stenting and Angioplasty With Protection in Patients at High Risk for Endarterectomy trial led to United States Food and Drug Administration approval of CAS for the treatment of patients with symptomatic carotid stenosis. In contrast, recent trials, such as the Stent-Protected Angioplasty Versus Carotid Endarterectomy trial and the Endarterectomy Versus Stenting in Patients with Symptomatic Severe Carotid Stenosis trial, (re)fuelled the debate between CAS and CEA. In the Stent-Protected Angioplasty Versus Carotid Endarterectomy trial, the complication rate of ipsilateral stroke or death at 30days was 6.8% for CAS versus 6.3% for CEA and showed that CAS failed the noninferiority test. Analysis of the Endarterectomy Versus Stenting in Patients With Symptomatic Severe Carotid Stenosis trial showed a significant higher risk for death or any stroke at 3,0 days for endovascular treatment (9.6%) compared with CEA (3.9%). Other aspects-such as evolving best medical treatment, timely intervention, interventionalists' experience, and analysis of plaque composition-may have important influences on the future treatment of patients with carotid artery stenosis. Conclusions: CAS performed with or without embolic-protection devices can be an effective treatment for patients with carotid artery stenosis. However, presently there is no evidence that CAS provides better results in the prevention of stroke compared with CEA. (C) 2008 Excerpta Medica Inc. All rights reserved